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Introduction 



This report is a collection of algorithms developed by the 
author for the solution of integer programs. In Section 1, an 
algorithm for computing a one-dimensional knapsack function is 
presented. 

Section 2 consists of an algorithm for the knapsack problem 
where the constraint equation is a congruence. By first trans- 
forming the linear programming solution of the integer program 
into an equivalent knapsack problem, the integer solution may 
often be found by rounding the continuous solution. 

Section 3 consists of an algorithm for the general solution 
of integer linear programs. The continuous linear programming 
solution is found first. If the continuous solution is fractional, 
then a linear congruence is added as a constraint. A dynamic 
programming formulation of the problem is made and the optimal 
integer solution is found. 

The final algorithm in Section 4 is a branching algorithm, 
which produces an optimal integer solution to the problem by 
systematically changing the bounds on the variables. The method 
requires minimum excess computer storage over that required 
to solve the linear programming problem. In addition, dynamic 
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programming enumeration techniques, similar to those used in the 
previous algorithm, may be used to speed the calculations once a 
solution to a linear program is obtained. 



1. An Algorithm for the Computation of Knapsack Functions 

In this section, we present an algorithm for computing a one- 
dimensional knapsack function defined by: 



n 



F(x) = max { £ c,x. 

j=i J J 



n 



2 a.x. = x, x. a 0, x. integer} 
j_l J J J J 



where the c. and the a. are given constants. 

J J 

Knapsack problems have been studied in [5], [6], [7], [12]. 
See these references for physical applications. In this section we 
present an algorithm that is more general and simpler than that 
given in [7]. Both are modified dynamic programming algorithms. 
Ours, however, has the computational advantage of being a single- 
pass algorithm, while the algorithm in [7] requires the maintenance 
of a length grid to permit backtracking to find the values of the 
variables. 

A modified dynamic programming algorithm is also given in 
[12]. However, it is presented in a network context of considerable 
complexity and appears to have no computational advantage over [7]. 
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It is evident from the work presented here that the problem 
can be solved as a relatively simple enumeration with a very simple 
algorithm. The enumeration is for only feasible values and, thus, 
represents an additional computational advantage over that in [7] 
or [12]. The greater generality found here is that the constraint 
can be an equality constraint with no restrictions on the constants; 
the algorithms in [7] and [12] can solve equality constraint problems 
provided there is a variable with unit coefficient in the equality. 

We define the knapsack problem as: find x^, j = 1 , . . . , n that 

n 

maximizes £ c.x. 

j=i jj 

n 



when £ a.x. = L 

j*i jj 

x. ^ 0, x. integer, 



(i) 



where each c. is a positive number, each a. is a positive integer, 
J J 

and L is a feasible positive integer. In the usual knapsack problem 
the constraint in (1) is an inequality constraint. We consider the 
more general problem with the equality constraint. In problems 
with an inequality we would simply have a slack variable. 

The one -dimensional knapsack function is defined from (1) as: 



n 



F(x) = max f £ c.x. 

j=i JJ 



Z) a.x = x, x. £ 0, x. integer }. (2) 

■ J J J J 
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If F(x) can be found for all values of x and in turn the x be found 

that produces the solutions, then (1) is solved when F(L) is determined. 

Following [7], the knapsack function can be written as 

F(x) - max [c. + F(x - a )] 

j:a. ^ x J J (3) 

F(0) = 0, 

for values of x that are feasible. In the algorithm to follow we will 
use only feasible values of x; thus we can write (3) in a simpler form 
than in [ 7] . 

One immediate solution to (3) can be found by finding a^ = min a^, 

for all i. This produces F(a ) = c . We then replace x by x - a 
J L r r r 

in (3), and we obtain 

F(x - a ) = max [c. + F(x-a -a.) ], (4) 

r . , ^ j r j 

i:a +a.£x J J 

r J 

which we substitute for the F(x - a ) term on the right side of (3). We 

r 

then can produce another immediate solution. We continue this way until 

F(L<} is found. We also keep track of which j produces the solution to 

determine the optimal x. at the end. We also only consider distinct a. 

J J 

values. If a. = a, and c. a c, , j ^ k, then take x, =0. This 

J k j k k 

entire procedure is contained in the following algorithm: 
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1. List the values of the problem as follows: 



n 



C 2 C 3 

a 2 a 3 



n 



n 



x =0 
J 

Set m = 0. Go to 2. 

2. Given the list, find a = min a. for columns in all sections. 

r J 

a>m 
J 

If a = L, go to 4. Otherwise set m = a and go to 3. 
r r 

3. Add a new section of columns to the list, if possible, as follows: 

Calculate a 1 = a + a and c 1 = c + c for t=l f . n. 
t r t t r t 

Add a column headed by t if: 

a) a‘ £ L. 

' t 

b) a 1 is not on the list. 

' t 

c) a 1 is on the list and has a corresponding c^ value that is 

t t 

smaller than c*. 

t 

d) Underneath the section added write the x. values from the 

section where m = a was found. Increase x by one for 

r r 

the new section. 



Sections after the first may be deleted after they are no 
longer needed. 

Go to 2. 
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4. The problem is solved with solution c . The values of the var- 
r r 

iables are found below the section where a^ = L appears; 

increase x by one, 
r 

This completes the algorithm. The algorithm may be modified 

as in [7] if L is much larger than the a.. The algorithm may be 

J 

modified appropriately to handle the discounted problem [12], where 
a . 

a ^ multiplies F(x-a.) on the right hand side of (3). 

Example: 



Max 2x^ + — x^ + 5x^ 
when 2x + 3x + 4x = 6. 

1 u J 

x ^ 0, x ^ 0, x ^ 0, all integer. 

1 ^ J 

We list 



_L_ 

2 

2 



_2_ 

10 

3 

3 



2 

5 

4 



x.=0 

J 

and min a. =2. We add the section 
J 



_ 2 _ 

iA 

3 

5 



2 

7 

6 



(5) 



( 6 ) 



- 6 - 



We obtain min a =3 in (5). No new section is added. 

j 

We obtain min a^=4 (5). No new section is added. 

The solution is now evident in (6), where a. = 6. We thus have 
F(6) = 7, x^ = 1, x^ = 1 as optimal. 

2. Knapsack Solutions in Integer Programming 



section, we present 


an algorithm for 


solution to the problem 


m 




m 




minimize 2 


c .x. 


when 2 ax. ~ 


b (mod D). 


i=l 


J J 


j=i J J 



This problem solution is useful for solving integer programs. 

We are interested in the integer programming problem: 

find X=(x, , x n , . . . , x ) that 
12 n 

minimizes CX 

when AX = B (7) 

x. ^ 0 and integer, j = l, . . . , n, 

where C is an n-vector, B is an m-vector, and A is an mxn matrix. 
We assume that the components of C, B, and the elements of A are 
integers . 

Gomory [8] has presented a method that may solve (7) under 
certain circumstances. He solves an equivalent knapsack problem by 
dynamic programming to round off the linear programming solution to 
(7). Shapiro [11] presents another dynamic programming algorithm. 
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In this section, we present a simple algorithm that solves the 

equivalent knapsack problem. We also present a simpler equivalent 

knapsack problem than in [ 8] or [ 1 1] . 

The linear programming solution to (7) transforms (7) to 

minimize d + Z c.x. 

jeG J J 



when + Z a.x. = <* 

G . 7 ? J J 0 
jeu 



( 8 ) 



x. ^ 0 and integer, j = l, . . . , n, 

where X^, = {x. J ieG}, £ 0, c. ^ 0, G is the set of indices of 

the basic variables and G is the set of indices of the non-basic variables. 

The vectors cr (j = 0 and jeG) are column vectors. If is all integer 

then x. = 0 (jeG), X = a is an optimal integer solution. If any of the 
J GO 

are fractional, then the problem (8) is reduced to an equivalent 
knapsack problem 



minimize 



c .x. 



jeG JJ 



when £ a.x. = b (mod D) 

jeG J J 



(9) 



x. s 0 and integer, 

where the constraint in (9) is a single equation (in this case a congruence) 



and D is the determinant of the coefficients of the x., (ieG) from (7) 

(or a reduced value as seen below). We also assume that the c , a 

J j 
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and b are integers. The c. in (9) are the c. in (8) multiplied by D. 
We develop the congruence in (9) by finding the equation in (8) (including 
the objective function) that has any constant (i. e. , considering d, c., 

<2., and <2q)> written in fractional form, having the property that its 
numerator and denominator are relatively prime. In our case the 
denominator is D. Gomory [9] shows that if we find a constant with 
this property, then the congruence taken from the equation where the 
constant appears generates the congruences developed from the other 
equations. This is due to the group properties of the fractional parts 
of the constants in (8). The cutting plane developed from the congruence 
represents the best cutting plane, since any integer solution to the 
congruence produces integer solutions for the x,,( ig G) in (8). 

While the search for a constant with the property that numerator 
and denominator are relatively prime may appear to be tedious, we 
have developed a simple and rapid way to find the equation containing 
the constant. We use the revised simplex method to obtain the linear 
programming solution to (7). Thus, the inverse matrix is available 
to develop the constants in (8). In solving (7), we carry along the value 
of D as the product of the pivot elements. In an obvious way the 
desired equation is found by finding the greatest common divisor of 
the non-zero numerators of each of the rows of the inverse matrix. We 
stop when the greatest common divisor is unity. If none of the rows 
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produce a greatest common divisor of unity, we select the row with 
minimum greatest common divisor. The greatest common divisor of 
the greatest common divisors found for each row also divides D. 
Accordingly, we then produce a smaller value of D by dividing. We 
use an enlarged inverse matrix to consider the objective function also. 

Before entering the above process we first examine the ot^ and d 
using the Euclidean algorithm. We stop when we reach a fractional 
constant having unity as the greatest common divisor of the numerator 
and denominator. If the required row is not found, then we utilize the 
inverse matrix as above. 

We use as an example the problem given in [2], where it illus- 
trates Gomory's algorithm. 



min 






4x + 


5X 2 




when 


3X ! 


+ 


X 2 ' 


x 3 = 2 






*1 


+ 


4 x 
* X 2 


X 4 = 5 


(10) 




3x_ 


+ 


2x - 


x = 7 






1 




2 


5 






x. 


£ 


0 and 


integer. 
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The continuous solution gives us 



min 



112 7 , 11 

— + To X 4 + To X 5 



x l + To X 4 



10 X 5 = 10 



18 

10 

8 



_3_ _L x 

X 2 10 X 4 + 10 X 5 ' 10 



(11 



where D = 10. 



3 11 42 

X 3 + 10 X 4 10 X 5 " 10 



Considering d = 



112 , 18 8 42 v 

To " 5 * 0 = ( To 3 To 3 To we see that none have 



numerator and denominator relatively prime. The inverse matrix is 






TT T-T- 0 



= 2 


4 


10 


10 


3 


-1 


10 


10 


-3 


11 


10 


10 


-7 


-11 


10 


10 






44 1 



where the bottom row produces the coefficients in the objective function. 
In the first row we consider the greatest common divisor of (-2,4), 
which is 2. The first row produces the first constraint equation in (11). 
In the second row we consider the greatest common divisor of (3, -1), 
which is unity. Thus, the required congruence may be obtained from 
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the second constraint in (11). Similarly, the required congruence 

may be obtained from the third constraint and the objective function. 

Using the second constraint, we obtain the congruence 

7x. + x r s 8 (mod 10). 

4 5 

The equivalent knapsack problem becomes 

minimize 7x^ + llx^ 

4 5 

when 7x^ + x^ = 8 (mod 10) 

x , x^ ^ 0 and integer. 

In general, once the required equation in (8) is found; e. g. , 

x. + 2* or,, x. = or for a single ig G 

1 -7 1J J l0 

jeG 

(or for the objective function), the congruence is found as in Gomory 
[8] as 



where a. . 

ij 

we obtain 



2 oi., x. a a (mod 1) 

jeG ,J J 10 



is the fractional part of a.,. 

ij 



Multiplying through by D, 



2 a . x. ~ b (mod D) 
je Ct J J 

where a. = a . D, b = a D. 

J ij i0 
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We are interested in the solution to the problem 



n 

min L 

j=i 



c . x. 
J J 



n 

when a. x. = b (mod D). 

j=i J J 

x. ^ 0 and integer. 

We can assume that no value of a. appears more than once (if not, 
can select the one with smallest c. value). 



Writing 



n n 

F(y) “ min I 2 c.x. £ a.x. = y (mod D) 

L j-i j j j=1 j j J 



We obtain the recursion as in [ 7] 

F(y) = min ( 4- F(y-a^) ) 

with F(0) = 0. The arguments of F are taken modulo D. 
We can write (12) as 



F(y) = c + min (c. - c 4 F(y-a.) ) 
r . j r i 

J 

where c 4 min c.. Equation (13) has an immediate solution for 

r j J 

y=a , and we obtain F(a ) = c . We then write (13) as 
r r r 

F(y-a ) = c 4- min (c. - c + F(y~a - a.) ). 
r r j j r rj' 

We substitute (14) for the F(y-a ) term on the right side of (13) 

r 



and readily produce another solution to some F(a.). We stop when 

J 
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we 
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(13) 



(14) 



we have achieved a solution for F(b). We also keep track of the 
variable j, which produces each solution, and then back-track to 
find the x.. This entire procedure is contained in the following 
algorithm: 

1. List the values of the problem as follows: 

1 2 3 . . . n 



T 



c. . . . c 
3 n 



a . . . a 
3 n 



Go to 2. 



2. Given the list 



3 . . . n 



n 



^ ~ • • * 3 , 

3 n 



find c = min c. for all unmarked columns. Set F(a ) = c and 
r j r r 

I{a ) = r, where a and r are the corresponding elements to in 

the column. If a = b, go to 4. Otherwise, mark the column if it is 
r 

one of the first n. Go to 3. 



3. Add columns to the list, if possible, as follows: 

Calculate a J = a + a (mod D) and c 1 = c + c for t = l, 
t r t t r t 

Add a column headed by t if: 

(a) ^ as not k een found. 



. , n. 
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(b) is not on the list. 

(c) aj^ is on the list and has a cor responding c value that is 
larger than c|. 

Delete columns (beyond the first n) having elements a| 

(d) after F(a|) is found. 

(e) after a new a| is found as in 3(c). 

Go to 2. 

4. The problem is solved with solution F(b). The values of the 
variables are found as follows: 



(a) 


Set x. = 
J 


0, 


j = l, . . . , n and y=b. Go to 4(b). 


(b) 


Look up 


i(y) 


= j. Set x = x + 1. Go to 4(c). 


(c) 


Set y s 


y - 


a. (Mod D) . If y=0, go to 5. Otherwise, go 




to 4(b). 







5. End. The final x. values are the required ones. 

J 

This completes the algorithm. If solutions are desired for all 
values of y, we continue the algorithm instead of stopping when b is 
reached. In addition, the algorithm may be changed into a single-pass 
algorithm, as in the following sections. 

In the example above, 

Min 7x, -f llx^ 

4 5 

when 7x -1- x s 8 (mod 10) 

4 5 

x , x s 0 and integer, 

4 5 
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we list 



4 5 

7 11 

7 1 

Thus, 7 = min c^, F(7) = 7, 1(7) = 4. The list becomes (marking 

the first column) 

4 5 4 5 

7 11 14 18 

7 14 8 

Thus, F(l) = 11, 1(1) = 5. The list becomes 

4 5 4 5 5 

7 11 14 18 22 

7 14 8 2 

F(4) = 14, 1(4) = 4. The list becomes 

4 5 4 5 5 

7 11 18 22 25 

7 18 2 5 

>!« * 

F( 8) = 18, 1(8) = 4. We stop. 

Backtracking, 1(8) =4, x 4 = 1. 

1(1) = 5, x = 1, end. 

5 

The solution is F(8) = 18, x = 1, x =1. 

4 5 
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We have used the above procedure to solve integer programs 
arising as covering problems. The method usually produces an 
immediate integer solution. If the knapsack solution does not produce 
feasible (positive) integers to the problem, we have always been success- 
ful after adding a single cut. This was done on approximately 25 
problems where there was a maximum of 2000 variables and 150 
equations. 

3. An Algorithm for Integer Solutions to Linear Programs 

In this section, we present an algorithm for the solution of 
integer linear programs where the variables have upper bounds. The 
continuous linear programming solution is found first. If the continuous 
solution is fractional, we develop a linear congruence that is added as 
a constraint. The optimal solution is then found after a dynamic pro- 
gramming formulation of the problem is made. 

We are interested in the integer programming problem: find 

X = (x , . . . , x ) that 
* 1 n 

minimizes CX 

when AX = B (15) 

0 £ x. £ b., x. integer, j = l, . . . , n, 

J J J 

where C is an n-vector, B is an m-vector, A is an mxn matrix, 
and the b. are integers (any or all of the b. may be infinite). We 



- 17 - 



assume that the components of C, B, and the elements of A are 



integers. 

The solution of (15} includes the case where the x. are restricted 

j 

to being either zero or one. 

We use a bounded variable linear programming technique as in 
[3] to obtain the equivalent problem: 



minimize 



d + £ 



c . x. 



■ r J J 
J€ u 



X + E ot. x. = of , 
G j e G J J 0 



( 16 ) 



0 £ x. £ b., x. integer, j=l , . . . , n, 

J J J 

where the vector X = ( x I i$G} , G is the set of indices of the 

G i 5 

basic variables, and G is the set of indices of the non-basic variables. 

Further, some of the non-basic variables may be at their upper bounds 

in the continuous solution of (15). For those variables we would have 

their corresponding c. values as non -positive . We also have c. ^ 0 

J J 

for non-basic variables that are at the zero value. For ease of compu- 
tation, we make the transformation x[ = b. - x. for those non-basic 

J J J 

variables at their upper bound. Thus, we may assume a form like 

( 1 6) with all c. ^ 0 and all non-basic variables at zero values. The 
J 

vectors a ^ (j=0 and jeG) are column vectors. The components of 

are non-negative. If is all integer, then x. = 0 (jeG), = ot Q 

is an optimal solution (if any of the x., jeG are really the x*. under 

J J 
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the transformation, we would have x. = b.). If any of the 

J J 

fractional, problem ( 1 6) is converted to the problem: 



minimize 2) 
e G 

when £ at. x. £ oi 



c. x. 

1 F J J 

je o 



jeG J J 



“o are 



(17) 



Jj a. x. = b (mod D) 
jeG J J 

0 ^ x. ^ b., x. integer, jeG 

The inequality constraints in (17) are obtained from (16) by dropping 

the x., ieG. D is the determinant of the coefficients of the x., ieG 
l i 

from (15), The value of D may be found as the product of the pivot 
elements during the simplex procedure for obtaining the continuous 
solution of (15), The linear congruence in (17) is a single congruence 
where the a. and b are integers. The congruence results from the 
requirement that all x., ieG, be integers. We develop the congru- 
ence, as in the previous algorithm, by finding a constant among 
d, Cy 01 y and 0^, written in fractional form as m/D, having the 
property that m and D are relatively prime. We previously showed 
how to find the equation of ( 1 6) with the required constant. We then 
develop the congruence from the equation. Most linear programming 
problems have certain group properties [9], so that any integer solu- 
tion of the congruence produces integer values for the x., ieG. 
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We proceed zo develop the dynamic programming formulation 



of (17). Writing 



min I 2 c .x. 


I £ a.x. £ a, 


£ a.x. = x (mod D), 0 £ e.x. £ p 1 


L; P J J 
J€ G 


' ■ r J J 
jeG 


• J J J J ^ 

JeG 



we obtain the dynamic programming recursion 

F(x, o', P) - mm (c. + F(x-a., a- a\, P-e.)) 

• Z"' J J J J 

jeG 

F(0,a, P) = 0 for ot ^ 0, p ^ 0. 



(18) 



The x argument of F is taken modulo D. The vector e^. is com- 
posed of a unit element with the other elements zero. The unit ele- 
ment corresponds to the unit coefficient of x. in x. £ b.. We also 

J J J 

assume that only feasible values of x, a, and P are used in (18). 

One immediate solution to (18) (tan be obtained by finding 

c = min c.. This produces F(a , a , e ) = c . We then replace 
r j rrrr 

x by x-a , ot by ct-at , and p by P-e in (18), and we substi- 
r r r 

tufe the result for the F(x~a , ot-ot , P-e ) term on the right side 

r r r 

of (18). We then can produce another immediate solution. We con- 
tinue this way until F(b, a, P) is found where 0 £ a - a £ b and 

0 G 

P £ b— where the vector b^ = {b.ligG} and the vector 
G G l 1 J 

b— = [bj igG}. This entire procedure is contained in the following 
algorithm: 
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1. Suppose the indices in G are (l,2,...,m), list the values of 
the problem as 




j 

Go to 2. 

2. Given the list, find c = min c. for all unmarked columns in all 

r J 

sections. If a = b and 0 £ or - a ^ b . go to 4. Otherwise, 
r * 0 r G 

mark the column and go to 3. 

3. Add a new section of columns to the list, if possible, as follows: 

a) calculate c\ = c 4- c., ot\ = a 4- a a* = a + a. (mod D) 

J r JJ r JJ r J 

for all values jeG (i. e. , values j, c., a., a, are taken from 
the list in step 1). where x^ < b^ for j / i and where 
+ 1 < b^ for j~r for the section containing the newly- 
marked column. 

b) add the column headed by j in the new section with values 

c 1 . , a 1 and a 1 .. 

J J J 

c) underneath the section added write the x„ values from the 

J 

section containing the newly marked column. Increase x^ by 
one for the section. 
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When columns in new sections become marked, they may 

be deleted after the calculations are made and the new section 

is added to the list. A new column may already be on the list. 

It need not be added if the x. values in corresponding sections 

would coincide (after increasing the x. values by one for the 

j heading values of the two columns). Go to 2. 

4. The integer problem is solved with objective function value 

d + c . The values of the basic variables in (16) are 
r 

= cv - a. . The values of the non-basic variables are found 
G 0 r 

below the section where a = b appears; increase x by one. 

r r 

This completes the algorithm. An integer solution is sure to 

be found because all integer solutions to the congruence in (17) are 

systematically produced in order of increasing objective function 

value. The algorithm may be simplified by calculating a\ only when 

a! = b by using the proper x. values and the a. from step 1. 

J J J 

Furthermore, if any of the components of X in 4 are fractional, 

G 

then we have a multi-dimensional group. We simply add another 
congruence and treat a„ and b in (17) as vectors and continue the 
algorithm in step 2. 
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Example: We take the problem given in [ 1] 



We 



minimize 5x^ + 


?X 2 


+ 


10x 3 + 


3x 4 + 


*5 






when 


x i - 


3X 2 


4- 


5 x 3 + 


X 4 “ 


4x 5 


£ 


2 




- 2x i + 


6x 2 


— 


3x 3 - 


2x 4 + 


2X 5 




0 




- 


X 2 


+ 


2x 3 - 


X 4 


X 

5 




1 




0 ^ X. 
J 


£ 1. 


x . 
J 


integer , 


j = 1.... 


.,5. 






introduce surplus 


variables 


x 6 * 


o, 


X 

-J 

IV 

o 


and x Q s 
o 


0 (i. 


e. 


> 



with a> as upper bounds) and find the continuous solution and equiva- 
lent problem; 



... „ 93 

minimize 9 + 



1 56 42 

+ — x 4 + T X 5 



24 137 

+ ~ x 7 + — x 8 



7 

9 X 1 



28 13 1 

-x. + X_ + x, + — X 



9 4 9 5 



9 7 



3 X 8 3 



2 

“ 9 X 1 



8 4 

+ x 3 ' 9 X 4 ’ 9 X 5 



1 

9 X 7 



3 X 8 3 



- 9 X 1 + X 2 



7 1 

9 X 4 + 9 X 5 



2 1 1 
-—X _ + TX n “ T 



9 7 



0 £ x. ^ 1, x. integer, j = 1, . . . , 5 
J J 

0 £ X., j = 6, 7, 8. 
j 

We have D = 9. The second equation produces the congruence 



3 8 3 

(19) 



7x^ + x^ + 5Xj. + 8x^ + 3Xg = 6 (mod^) 
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which is added as a constraint. Multiplying through by 9 where appli- 



cable, we list 




( 20 ) 



For convenience we include at the left of the list the objective function 

value, the right-hand sides (multiplied by 9) for the equalities in 

(19), and the right-hand side for the congruence. We have 

24 = min c.; i. e. , c = 24, with r = 7. We add the section 
J r 




( 21 ) 
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We mark the 7 column in (20). We have min c. ~ 42 from (20^; 
we add the section 



1 


4 


8 


135 


198 


178 


6 


-15 


10 


-6 


-12 


-10 


-3 


-6 


4 


3 


6 


8 


X 5 


= 1 





( 22 ) 



We need not add a column headed by 7 in (22) because it would dupli- 
cate the 5 column in (21) (the use of either column would require 
= 1 and - 1). We do not add a 5 column in (22) since x^ 

would be at its upper bound in (22). We mark the 5 column in (20). 



We have min Cj = 48 in (21). We add the section 



1 


4 


5 


7 


8 


141 


204 


90 


72 


185 


-5 


-26 


15 


3 


-1 


-4 


-10 


-6 


-3 


-8 


-8 


-11 


-3 


-6 


-1 


5 


8 


3 


6 


1 


X 7 


= 2 









(23) 
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We mark the 7 column in (21). 



We have min c. = 66 from (21). 
J 



We add the section 



1 


4 


8 


159 


222 


203 


-7 


-14 


11 


- 7 


-13 


-11 


-5 


-8 


2 


2 


5 


7 


X 1 = 


1, x 5 


= 1 



(24) 



We mark the 5 column in (21). The solution is now apparent in the 
7 column in (23). We have the optimal solution = 3, 
x, = (3 - 3)/9 =0, x 3 = (6 + 3)/9 =1, x 2 = (3 + 6)/9 = 1; 
the objective value is 9 + 72/9 = 17. 



4. The Solution of Integer Programs by Bounded Variable Techniques 



In this section, the integer programming problem is solved by 
linear programming using upper and lower bounds on the variables. 
The bounds are changed systematically until the optimal integer 
solution is produced. The method results in a branching process 
that requires minimum excess computer storage over that required 
to solve the linear programming problem. In addition, dynamic pro- 
gramming enumeration techniques are used to speed the calculations. 
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The basic method may be applied to problems where not all the varia- 
bles are integer restricted. 

The branch and bound method of Land and Doig [10] is to solve 



(15) first as a linear program to obtain the continuous solution 
(z^, x^, . . . , x^). If the continuous solution is all integer, the prob- 



lem is solved. Otherwise, the value of the objective function z 



0 



provides a lower bound on the value of the optimal integer -solution; 

some variable is selected whose value x^ is not an integer. 

Thus, integer x^_ must satisfy either x^ £ [ xj^ ] or ^ ^ 

where [x^] is the greatest integer less than xj\ The procedure 

initially involves the solution of two problems: the linear program 

min z subject to x^_ = [x^], in one case, and x^ = [ xj^ ] + 1, in 

the other case, added to the constraints of (15). 

11 0 1 

Suppose the solutions are (z , x^ ,...,[ x^_ ],.. . x^) and 

2 2 0 2 1 2 0 
(z , x^,...,[x^] + l,...,x^) with z , z ^ z . The problem 

1 2 

solution with smaller z or z is investigated first (while the 
other problem is stored for possible later use). Suppose it is z\ 

If the 7 } solution problem is all integer, then (15) is solved. Other - 



wise, some variable x is selected whose value x is not an 

r r 

integer. Thus, three new problems are solved: the linear programs 

min z subject to x, = [ x 1 , x = [ x ] , in the first case, 

J k k J r r J 

0 1 0 
x = [x ], x = [x ] + 1, in the second case, and x = [x ] - 1, 
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in the third case, added to the constraints of (15) (assuming 

o£] * °>- 

The process is then repeated. There are four problem solutions 
that are investigated. The method continues in the above fashion until 
an integer solution is achieved. 

This entire procedure enumerates possibilities for the solution 

0 

to (15) m a directed tree. The rooted node corresponds to z and 
has directed branches to nodes corresponding to the solution of the 
linear program (15) with x^ set at specific integer values, each rep- 
resenting a single branch. From each node the same procedure is 
repeated. The Land and Doig method develops the tree by branching 
from the terminal node that has minimum objective value. This will 
then limit the potentially large size of the tree. 

The Land and Doig method works well when there are few 
variables in a problem. The method does not work well when the 
number of variables is large due to the possibly immense storage 
requirements for saving the problems that represent the terminal 
nodes. 

In this section, we develop a different branching procedure that 
requires a minimum of excess storage beyond that required to solve 
the linear programming problem. In addition, dynamic programming 
enumeration techniques are used to reduce the number of branchings 



required. 
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We solve (15) as a linear program to obtain the continuous solu- 
tion. For a new branching procedure several observations are pos- 



sible first: 

(a) The Land and Doig method may be simplified by solving the 



(15). This is done also for any terminal node. Thus, any ter- 
minal node would have a maximum of two branches leading to 
the next level of the tree. This procedure tends to reduce the 
number of nodes that we need to investigate. 

(b) The procedure may be further simplified when a feasible integer 
solution is obtained at any node. The objective value z = x^ 
represents an upper bound to the optimal z. A good strategy 
is to investigate a single branch of the tree until [z ] + 1 ^ x^. 
We then backtrack in the tree until a node is reached that has 
had only one branching. This is achieved in the problem by re- 
moving the bounds beyond the node and solving the resultant 

linear program. Then the second branch is made (i. e. , in 

r 

back-tracking we reach a node that has only x > [x ] + 1 as 

K xC 

a branch. We remove all bounds beyond the node and solve the 
linear program starting from the latest transformation of the 





^ [ x v ] + 1* * n other case, added to the constraints of 

xC 



equations. We then 




and continue on a single 
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branch). This procedure has the advantage of developing only 
one level of the tree and storing only the current bounds. If 
any feasible integer solution is found with objective value less 
than Xq, we use the new solution as an upper bound to the 
optimal value of z. Solution is achieved when it is impossi- 
ble to branch any further. The current upper bound solution 
is then optimal. 

(c) Feasible integer solutions may be found using a dynamic pro- 

gramming enumeration similar to that given in the previous 
section. This can be done at any node. In performing the 
enumeration it can be noted when will be surpassed. The 

enumeration can then be stopped and the back-tracking pro- 
cedure in (b) instituted. 

(d) The enumeration procedure can be instituted at the root node, 
thus producing the optimal solution. However, if D (the prod- 
uct of the pivot elements in solving (15) by the simplex proce- 
dure [3] is large, then the dynamic programming enumeration 
may require excess storage. However, the value of D tends 
to reduce as more bounds are made. Thus, the dynamic pro- 
gramming enumeration will require less storage as higher 
levels in the tree are reached. 

(e) The selection of the non-integer variable in the branching may 
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be critical in reducing the number of nodes. When D is large, 
we use the criteria of [4], When D is small enough, we use 
a dynamic programming iteration. It is possible to stop the 
enumeration before completion and obtain infeasible integers 
for the variables. We have then branched on the variable that 
is most negative. 

We have used the two-phase simplex method to solve (15) to- 
gether with the bounded variable technique in [3]. Thus, we main- 
tain m equations and have a solution where some variables will be 
basic, some will be zero, and some will be at their upper bound. 

If all b. are infinite, the method reverts to the usual one. In the 
J 

branching procedure some variables will have non-zero lower bounds. 

We are interested in finding the solution criteria to (15) with 
bounds given by 0 £ a. £ x. £ b. <5 b., j = 1, . . . , n. Suppose 
we achieve solution values for the x., satisfying the bounds plus 
the constraint equations, and have obtained the following canonical 
form [3] from the constraint equations in (15): 

n 

x . 4* 2 a. . x = d. i = 1 , . . . , m 

1 j=m+l lJ J 

(25) 

n 



-z + 2 

j=m+l 



c . x. 
J J 



-z 



0 * 
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The variables x., i = 1, . . . , m, are the basic variables, and 
1 

the variables x., j = m + 1, . . . , n, are the non-basic variables, 

J 

We have the following: 

Theorem: Any values of x^, j = 1, n satisfying the constraint 

equations in (25) and 0 ^ a. £ x. £ b. £ b., j = 1, . . . , n 

J J J J 

are an optimal solution for min z, with objective value z^ , 
if c. ^ 0 for variables at their lower bound a. and 



J J 

c. ^ 0 for variables at their upper bound b.. 

J J 

Proof: Any increase in the variables at their lower bound or any 

decrease in the variables at their upper bound can only 
increase z. 

The theorem produces the conditions for optimality. To improve 

a feasible solution that is not optimal, we can increase a non-basic 

variable x. at its lower bound when c. < 0, or can decrease a non- 
J J 

basic variable x. at its upper bound when c. > 0. Thus, when a 
J J 

feasible solution to the linear programming problem with upper and 
lower bounds can be found, we can use a variant of the usual simplex 
procedure to achieve optimality. 

After solving (15) as a linear program, we impose the condi- 
tion x^ £ [x^] or ^ [xj^] + •*" ^ or a basic variable x^ with its 
value x^ as fractional. The solution for the linear program is then 
infeasible. We next place x^ at either the upper bound [ x^] or the 
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lower bound [x^] + 1; add an artificial variable equal to the amount 

0 0 

of the infeasibility produced (x^ - [x^], in the first case, or 

r 0 

[x^J + 1 - x^, in the second case); and solve the new bounded var- 
iable problem by the two-phase procedure. If We obtain a solution 
to the new problem that is fractional, we impose the additional con- 
dition x £ [x ] orx £ [x^l + 1 for a basic variable x with 
r u r J r r J r 

its value x"^ as fractional. We then solve the new problem main- 
r 

taining the constraint on x^. We continue this process until it is 
necessary to backtrack, which is done by simply removing the bounds 
on the required variables and solving the linear program to return 
to a previous problem. Note that bounds on the same variable may 
appear again; the initial bounds will automatically be satisfied. 

This entire procedure enumerates possibilities for the solution 
to (15) in a directed tree. The rooted node corresponds to z^. We 

only investigate one branch to the next node corresponding to the 

0 

solution of (15) with, say, x ^ [ x 7 ] I 1 representing the branch. 

K K 

From each node the same procedure is repeated, investigating one 
branch until we backtrack to a previous node that has had only one 

branching. For example, we would branch to a node corresponding 

0 

to the solution of (15) with £ [x ] representing the branch when 

is. iC 

we backtrack to the rooted node. Any further backtracking to the 
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rooted node would end the problem. This procedure is contained in 
the following algorithm: 



Define S(x^, x^, . . . , x ) as the current upper bound solution, 

which represents a feasible integer solution to (15) with objective 



value Xq. 



th 



Define 1(D) as the index of the D — bounded variable. 

th 

Define N(D) to show the number of branchings for the L — 
bounded variable (Initially N(D) = 0). 



Define B(D) to show the bound on the 

th 



th 



bounded variable. 



Define G(L) = 0 to indicate the L — bounded variable is £ B(L) 

th 

and G(L) = 1 to indicate the L — bounded variable is ^ B(L). 

1. Set D = 0 and take S(x^, x^, . . . , x^) as a feasible integer 
solution to (15). If none is apparent, take x^ as infinite. Solve 
(15) as a linear program. If the solution is all integer, the prob- 
lem is solved. Otherwise, go to 2, maintaining the canonical 
form of the solution to (15). 

2. Set L = L + 1. Go to 2(a). 

(a) Select a basic variable that is fractional. Suppose the 

variable selected is with value x^, set I(L) = k, 

N(L) = 1 and go to 2(b) or 2(c). 

(b) Set B(L) = [ x^j and G(L) = 0. Go to 3. 

(c) Set fc(L) = [x°] + 1 and G(L) = 1. Go to 3. 

- 34 - 



3. Solve the linear program using the maintained canonical form 

with the new bounds on the variable where k = I(L). One 

of the cases holds: 

(a) If the solution produces an objective value z with 
[z] I 1 ^ go to 4. 

(b) If the solution produces an objective value z with 

[z] + 1 < Xq and the solution is all integer, redefine 

S(Xq, x^, . . . , x^) as a new feasible integer solution 

where x A = z, x, , . . . , x is the new solution. Go 
0 1 n 

to 4. 

(c) If the solution produces an objective value z with 
[z] -f 1 < Xq and the solution is fractional, go to 2. 

(d) If the problem has an infeasible solution, go to 4. 

4. Set LL = L and go to 4(a). 

(a) If N(L) = 2, go to 4(b). Otherwise, N(L) = 1; go 
to 4(c). 

(b) If L = 1, the feasible solution S(x , x , . . . , x ) is 

0 1 n 

optimal. Stop. Otherwise, set N(L) “ 0, L = L - 1 , 
and go to 4(a). 

(c) Solve the linear programming problem starting from the 
current canonical form with the bounds B(L), 

B(L + 1), . . . , B(LL) removed. If K = I(L) and x 

K 
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does not become a basic variable, then the solution is 



non-unique; is then made a basic variable. Go to 

4(d). 

(d) If G(L) = 0, set B{L) = B(L) + 1, N(L) = 2, and 
go to 3. 

Otherwise, G(L) = 1; set B(L) = B(L) - 1, N (L) = 2, 
and go to 3. 

This completes the branching part of the algorithm. Good 
selection rules for which basic variable to use in 2(a) and the choice 
of 2(b) or 2(c) are given in [4], The dynamic programming enu- 
meration, which speeds the convergence in the algorithm, is carried 
out after the linear programming solutions are found in steps 1 or 3. 

We can illustrate the basic branching method, as given above, 
where the variables are arbitrarily picked for branching, and the 
branching is always taken with a greater than or equal sign first, for 
a problem [2] used to illustrate the Land and Doig procedure: 



Min 


4x l 


+ 


5x 2 


when 


3x l 


+ 


x 2 - x 3 “ 2 




X 1 


+ 


4x 2 - = 5 




3x l 


+ 


2x 2 -=<5=7 


V 


X 2’ 


X 3> 


x^, ^ 0 and integer 



( 26 ) 



The continuous solution produces the canonical form 



, 2 4 18 

x, + t7t x - ttX, = 

1 10 4 10 5 10 



3 x 1 8 
X 2 " 10 X 4 + 10 X 5 10 



3 11 42 

X 3 + 10 X 4 " 10 X 5 ~ 10 



7 11 112 

Z + — — -X , -f 7~X r = - — — 



(27) 



Since the solution is fractional, we arbitrarily select the bound 
Xj £ 2 added to (27). We then obtain the canonical form 



11 5 

X 2 + 4 X 1 ' 4 X 4 - 4 

11 1 3 

X 3 4 X 1 ' 4 X 4 4 

5 1 9 

X 5 2 X 1 ‘ 2 X 4 " ' 2 

11 5 25 

-z + — -X, + — X . = - — 

4 14 4 4 

where (z, x x 4> x g ) = (47/4, 2, 3/4, 19/4, 0, 1/2). 

We arbitrarily select x^ a 1 and solve (28) with x^ s 2 
x s 1 to obtain the canonical form 

x 3 ' 3X 1 - x 2 = - 2 

x. - x, - 4x_ = -5 
4 12 

x - 3x - 2x = -7 

D Yu 

-z + 4x^ + 5x^ = 0 



(28) 



and 



(29) 
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where (z ? x. x , x , x , x ) = (13, 2, 1, 5, 1, 1), which is all 
1 Z d 4 D 

integer and becomes the solution vector S(x^, x^ , . . . , x^). We ^ en 

remove the bound x^ ^ 1 and apply the simplex procedure on (29) (e. g. , 

z may be decreased by decreasing x^) and obtain (28) again. We then 

impose the bound x^ ^ 0 on (28) in addition to x^ ^ 2 and obtain the 

canonical form 

x + 4x_ x. = 5 

1 2 4 

x. -f 1 lx_ - 3x. = 13 

3 2 4 



(30) 

X 5 + 10x 2 " 3x 4 = 8 

-z - llx + 4x. = -20 
2 4 

where the solution is (20, 5, 0, 13, 0, 8). Since z > 13, we remove the 



bounds x^ £ 0 and x^ s 2 and solve (30) to obtain (27) again. We take 

x^ £ 1 and obtain the canonical form 

x 3 1 7 

X 2 2 X 1 ‘ 2 X 5 = 2 

3 1 _ 7 

X 3 " 2 X 1 " 2 X 5 2 



(31) 



X 4 + 5X 1 ' 2x 5 = 9 

7 5 -37 

-z - 2 x i + 2 X 5 = ~ 

where z, , x 2> x 3> x , x ) = (14, 1, 2, 3, 4, 0). Since z > 13, 

the problem is solved with solution z = 13, x = 2, x = 1, x =5, 

1 2 ’3 

x ^ = 1> - 1. The procedure is exhibited in the following tree. 
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0 



x 



1 




1 



£ 2 



4 



1 



x 



2 




2 



a 1 



3 



2 



Node 0: (112/10, 18/10, 8/10, 42/10, 0, 0) 

Node 1: (47/4, 2, 3/4, 19/4, 0, 1/2) 

Node 2: (13, 2, 1, 5, 1, 1) 

Node 3: (20, 5, 0, 13, 0, 8) 

Node 4: (14, 1, 2, 3, 4, 0) 



The advantages of this basic branching algorithm are apparent over 
the Land and Doig procedure even in this small example. As shown 
in [2], the usual procedure requires 5 nodes. Here we need only- 



basic variable again with a value that produced integers. Further, 
no additional problems are generated except that the canonical form 
from the original continuous solution is varied depending on the bounds 
on the variables. Each problem leads into the next in a natural fashion. 

The branching method outlined above will work well if feasible 
integer solutions can be found rapidly. We can achieve this by instituting 



4 nodes, In addition, we can obtain feasible integer solutions more 
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a dynamic programming enumeration, similar to that described in 
the previous section, after a fractional solution is obtained at any 
node. The equations at any stage are given by (25). We select one 
equation in (2 5) and develop a linear congruence as in [9]. Many 
problems have systems of equations, with certain group properties 
[9], so that any integer solution to the congruence produces integer 
solutions for all variables. In problems without this group property 
(i. e. , multidimensional groups), we simply add additional congru- 
ences when necessary. Suppose the congruence obtained from (25) is 
n 



L a. x. = d (mod D). 
i=m+ 1 3 3 



(32) 



We then add (32) as a constraint to (25) and enumerate all integer 

solutions to (32) in a dynamic programming format to minimize z 

subject to the bounds on the variables. We continue until we achieve 

feasible integer solutions. The values are then optimal from the form 

of the enumeration. To simplify the calculation using (26), we make 

the transformation y. = x. + a.; we then have 0 £ y. £ b l . = b. - a.. 

J J 3 3 3)3 

For variables with c . < 0 in the objective function of (25), we make 

the additional transformation y[ - b ! . - y. 

J J 

After these transformations a dynamic programming enumeration 
may be made. Using methods similar to that given in the previous 
section, we obtain the following algorithm for the solution to (25) under 
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the transformations above, where we take c. ;> 0, and (32) becomes 

J 

m' 



L 

j=l 

1 . Suppose the 
problem as 



a. y. = b (mod D). 
J J 



y. indices are j=l, 



(33) 



m‘, list the values of the 



1 


2 


’V | 

... m 


i 

c i 


C 2 


c 

m' 


a i 


a 2 


a , 

m' 


y j 


11 

o 





Go to 2. 

2. Given the list, find c = min c. for all unmarked columns in all 

r J 

sections. Mark the column. If a = b, go to 4. Otherwise, go 
to 3. 

3. Add a new section of columns to the list, if possible, as follows: 

(a) Calculate c! = c + c., a[ = a + a. (mod D) for all values 

J r J J r j 

j on the list in step 1, where x. < b! for j / r and where 

x 4- 1 < b 1 for j = r, for the section containing the newly 
r r 

marked column. 

(b) Add the column headed by j in the new section with values 

CJ! and a 1 ., 

J J 

(c) Underneath the section write the x. values from the section con- 
taining the newly marked column. Increase x^ by one for the section. 
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When columns in new sections become marked, they may be 

deleted after the calculations are made, and the new section is 

added to the list. A new column may already be on the list. It 

need not be added if the values in corresponding sections 

would coincide (after increasing the x. values by one for the j 

heading values of the two columns). Go to 2. 

4. Calculate the values of the basic variables in (25) using the y^ 

values under the marked section with y increased by one (i. e. , 

r 

the y. are transformed back to the x ). We have one of 3 cases: 

J j 

(a) If the basic variables have feasible integer solutions, the 
problem is solved. 

(b) If any basic variable has a fractional value, find another con- 
gruence from an equation with fractional value and treat 

a. and b as vectors in (33). Go to 2. 

J 

(c) If any integer value of a basic variable is infeasible, go to 2. 
This completes the algorithm. As an example, suppose we 

use the canonical form in (28). The congruence may be taken from the 



x^ equation and is 

X 1 + ^ X 4 ~ * (mod 4). (34) 

Making the change of variables y^ = + 2, y^ = x^, we obtain 

y 1 + 3y 4 = 3 (mod 4) (35) 
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from (34) and 



z 





(36) 



The enumeration consists in listing the indices for y, and y 

14 

plus the coefficients from (35) and (36) (multiply by 4 in (36)) as 



1 


4 


11 


5 


1 


3 


>— 1 

II 

o 




o 

II 

>> 





We obtain a possible solution in the 4 column with values z = 13, 

= 1, and y^ = 0, which produces =1, x^ = 2. Substituting in 

(28), we obtain x = 1, x = 5, x = 1. Thus, no further branching 
2 3 5 

would be necessary from node L If x , x , or x would be infeasible 

2 3 5 

with x^ = 1 and x^ = 2, we would continue the enumeration. 

In addition, if a solution vector S(x^, x, , . . * x ) is available 

0 1 n 

and the enumeration would produce an integer z ^ x^, we simply stop 
the enumeration and backtrack as in step 4 of the basic algorithm. 
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